Technological advancements have enabled a variety of changes to the nature and delivery of selection assessments (Rotolo et al., 2018; Woods et al., 2020). Interviews, in particular, have been radically transformed. Historically, interviews have been conducted in person or over a telephone and involved an interviewer asking the questions (Brenner et al., 2016). When high quality video conferencing software became widely accessible, conducting interviews via video conference software became increasingly common, but still included an interviewer. Currently, there is a trend toward having “one-way” interviews in which candidates use video recording software to record their responses to a set of interview questions and then send them to the organization (Guchait et al., 2014). In this variation, there is no interviewer per se, and these video recordings are evaluated by a rater at a later point in time (i.e., asynchronous or one-way video interviews).
A one-way video interview process typically involves a web link that is sent to prospective applicants inviting them to an online platform to complete the interview (Torres & Gregory, 2018). Once the invitation is accepted, applicants are prompted to record their responses to predetermined interview questions on camera and submit those responses via the online platform (Brenner et al., 2016). The recorded responses are reviewed later at the convenience of an evaluator, be that a hiring manager, recruiter, or artificial intelligence (Woods et al., 2020). A 2015 survey performed by Futurestep, a Korn Ferry company, found that 71% of the seven-hundred executives they surveyed use video-interviewing (i.e., both synchronous and asynchronous) at their organization (Futurestep, 2015). Further, a report by HireVue, a leader in interview technology, showed a dramatic increase in the number of AVIs conducted from 13,000 in 2012 to 2.5 million in 2016 (Greenfield, 2016) to over eight million by the end of 2018 (HireVue, 2018).
Given the increase in technology-integrated workplaces over the past decade and the rapid move to remotely delivered selection processes due to the COVID-19 pandemic, we can likely assume that these numbers have greatly increased. There are several asynchronous video interview promises that have fueled this increased adoption, such as the reduced costs in the form of travel expenses and time it takes to individually interview applicants, increased administration flexibility for both applicants and organizations, expanded applicant pools, increased efficiency, and improved interview standardization (Blacksmith et al., 2016; Brenner et al., 2016; Langer et al., 2017).
Although there are a number of reasons why organizations would adopt AVIs into their selection procedure, there are also perils that come with AVI adaptation that should be considered prior to implementation. Some of these perils include the potential for poor candidate reactions, limited communication, and skipping through candidate submissions (Suen et al., 2019). Furthermore, with over 100 online vendors offering access to their video interview solutions (Software Advice, 2022) organizations are not only faced with the decision on whether to implement AVIs, but they are also faced with the decision of how. During this panel discussion we intend to discuss the variety of design decisions left to the organization. The overarching goal of this panel is to spur discussion on the promises and perils of AVIs and how practitioners are conceptualizing these facets.
Panelist Biographies
Michael D. Blair, M.S., is a Senior Personnel Research Psychologist for the Assessment and Evaluation Branch of the U.S. Office of Personnel Management (OPM). He serves as the lead psychologist for USA Hire, OPM’s online assessment platform. He manages assessment strategy and content for USA Hire and large-scale assessment projects for OPM. He is the liaison between IT and HR teams working to design and implement assessment solutions. In that role, he is providing expert guidance as OPM works to implement an online digital interview technology using both asynchronous (recorded) and synchronous (live) methodologies. Michael also serves as an internal consultant within OPM drawing on his expertise in assessment strategy and technology. Michael has over 25 years of assessment experience including both the private and public sector. He was in the forefront of movements to use computerized testing, internet-based testing, and unproctored testing. Throughout his career, he has helped organizations redesign recruitment and selection processes to better leverage technology, introduce alternatives to inefficient and ineffective assessment strategies, and drive assessment innovation. He has authored articles, conference presentations, and webinars on topics including talent analytics, big data, assessment technology, internet-based testing, and the ROI of assessment. Michael is a current board member and former President of the International Personnel Assessment Council (IPAC). He earned his Master’s Degree from the University of Maryland.
Dr. Scott Highhouse is a Distinguished Research Professor and Ohio Regents Eminent Scholar in the Department of Psychology, Bowling Green State University. He has been named a fellow of the APA, APS, and SIOP. Scott is founding editor of the journal Personnel Assessment and Decisions, and currently serves as Chief Science Officer for SquarePeg Hires. Scott has served as Publications Officer on SIOP’s executive board (2009-2012), and served on the board of directors for the International Personnel Assessment Council (IPAC; 2012-2016). Scott’s primary areas of expertise are assessment and selection for employment, employer brand management, and human judgment and decision making.
Dr. Elliott Larson is an assistant professor at Baruch College, City University of New York. His research is focused on the development of fair, valid, and innovative employee selection systems. Through his research on test design and psychometric properties, he examines the predictive validity and group score differences of cognitive and non-cognitive assessments and the utility of these measures in high stakes testing situations. In addition, Elliott partners with organizations to implement evidence-based programs for identifying and developing talent and maximizing diversity. His work on the development and validation of assessments has been recognized with the 2017 Innovations Award from the International Personnel Assessment Council, the 2021 M. Scott Myers Award from SIOP, and the 2021 Human Resources Management Impact Award from SIOP and SHRM.
Questions for Panelists
The proposed 90-minute session was designed around the following questions posed to the panelists by the co-chairs.
- What are some of the key issues that organizations should consider when implementing asynchronous video interviewing in the hiring process?
- Design issues (e.g., sequencing, scope, integration, multiple takes, preparation time)
- Security / privacy issues (e.g., data access, video access, data retention)
- Content issues (e.g., competency libraries, question banks, templates)
- Psychometric issues (e.g., scoring options, consensus ratings, rater bias)
- What trends are you seeing in either the literature or in the field when it comes to AVIs?
- How have you implemented or seen AVIs implemented?
- What is the current state on the use of AVIs in your industry, segment, etc.? How do you see this current state changing over the next few years?
- What do you see are the major AVI-related research needs currently (e.g., user experienced, psychometric)?
- Do you anticipate these research needs to require a different approach or focus than historical/typical selection and assessment research? Why or why not?
- Is this a promise or a peril?
- Do you see implementing AVIs to help with the efficiency of a selection system as a promise or a peril?
- Do you believe that implementing AVIs has the potential to reduce costs for a selection system?
- Are there costs associated with AVI that are different from or go beyond typical selection systems that could end up costing an organization more?
- What are the pros and cons associated with the on demand, anytime, anywhere, self-initiated applicant scheduling of most AVI applications?
- Is the high standardization of AVIs a promise or a peril? For example, interaction between interviewer and candidate, ability to clarify questions, ability to ask probing questions.
- How much should we take into consideration the reported negative candidate reactions to AVIs?
References
Blacksmith, N., Willford, J., & Behrend, T. (2016). Technology in the Employment Interview: A Meta-Analysis and Future Research Agenda. Personnel Assessment and Decisions, 2(1). https://doi.org/10.25035/pad.2016.002
Brenner, F. S., Ortner, T. M., & Fay, D. (2016). Asynchronous video interviewing as a new technology in personnel selection: The applicant’s point of view. Frontiers in Psychology, 7, 863.
Futurestep. (2015). Futurestep Executive Survey: Video Interviewing Becomes a Mainstay; Companies are Implementing New Video Recruiting Tactics. https://www.futurestep.com/news/futurestep-executive-survey-video-interviewing-becomes-a-mainstay-companies-are-implementing-new-video-recruiting-tactics/
Greenfield, R. (2016). The rise of the webcam job interview: ’It was honestly pretty horrible. In Chicago Tribune. https://www.chicagotribune.com/business/ctcareers-webcam-job-interview-20161017-story.html.
Guchait, P., Ruetzler, T., Taylor, J., & Toldi, N. (2014). Video interviewing: A potential selection tool for hospitality managers–A study to understand applicant perspective. International Journal of Hospitality Management, 36, 90-100.
HireVue. (2018). The Ultimate Guide to Candidate Experience.
Langer, M., König, C. J., & Krause, K. (2017). Examining digital interviews for personnel selection: Applicant reactions and interviewer ratings. International Journal of Selection and Assessment, 25(4), 371–382. https://doi.org/10.1111/ijsa.12191
Rotolo, C. T., Church, A. H., Adler, S., Smither, J. W., Colquitt, A. L., Shull, A. C., … & Foster, G. (2018). Putting an end to bad talent management: A call to action for the field of industrial and organizational psychology. Industrial and Organizational Psychology, 11(2), 176-219.
Software Advice. Top Video Interview Software - 2022 Reviews & Pricing. Retrieved 2 February 2022, from https://www.softwareadvice.com/hr/video-interviewing-comparison/
Suen, H. Y., Chen, M. Y. C., & Lu, S. H. (2019). Does the use of synchrony and artificial intelligence in video interviews affect interview ratings and applicant attitudes?. Computers in Human Behavior, 98, 93-101.
Torres, E. N., & Gregory, A. (2018). Hiring manager’s evaluations of asynchronous video interviews: The role of candidate competencies, aesthetics, and resume placement. International Journal of Hospitality Management, 75, 86-93.
Woods, S. A., Ahmed, S., Nikolaou, I., Costa, A. C., & Anderson, N. R. (2020). Personnel selection in the digital age: A review of validity and applicant reactions, and future research challenges. European Journal of Work and Organizational Psychology, 29(1), 64-77.